1. Field of the Invention
The invention described herein relates to the methodology of forming a fused composite color image from source gray level images, with application to visualizing the fusion of video imaging in the infrared spectrum (e.g., thermal infrared) with video imaging in the reflective spectrum (e.g., visible).
2. Description of the Background Art
The advantages of image fusion of the visible and thermal infrared modalities have been studied for quite some time [S. Horn et al., “Fused Reflected/Emitted Light Sensors”, Proceedings of SPIE Infrared Technology and Applications XXVII, Vol. 4369, pp. 1-13, 2001]. Visible imaging (which can also include intensified Near-IR imaging) reveals reflective phenomenology complementary to thermal infrared imaging that reveals emissive phenomenology. There are obvious extensions to using reflective SWIR imagery as well. When viewed simultaneously, the visible and thermal infrared modalities can provide significantly enhanced situational awareness for a variety of applications to the dismounted soldier, maritime/naval operations, Unmanned Aerial Vehicles (UAVs) as well as law enforcement. The key issues for practical portable field applications are how to effectively visualize these two complementary modalities at video rates with sufficiently low power consumption and small form factor. One of the simplest visualization schemes is commonly referred to as “A+B” fusion, whereby the signal is combined from each modality in different proportions, or, each modality is represented by a different color and combined together accordingly. This can be conveniently implemented in optical overlay systems. However, such a visualization scheme is usually not suitable for many applications with major drawbacks being the lack of invariance to visible illumination changes and confusion between visible and thermal infrared contrast. A number of computational image fusion methodologies for the visible and thermal infrared wavebands have been developed to address some of these issues [L. van Ruyven A. Toet and J. Valeton. “Merging thermal and visual images by a contrast pyramid”. Optical Engineering, 28 (7): 789-792, 1989; D. Fay J. Racamato J. Carrick M. Seibert A. Waxman, A. Gove and E. Savoye. “Color night vision: Opponent processing in the fusion of visible and IR imagery.” Neural Networks, 10(1): 1-6, 199; P. Burt and R. Lolczynski. “Enhanced image capture through fusion.” In Proceedings of IEEE 4th International Conference on Computer Vision, volume 4, pages 173-182, 1993; J. Schuler M. Satyshur D. Scribner, P. Warren and M. Kruer. “Infrared color vision: An approach to sensor fusion.” Optics and Photonics News, August 1998; D. A. Socolinsky and L. B. Wolff, “Optimal grayscale visualization of local contrast in multispectral imagery.” In Proceedings: DARPA Image Understanding Workshop, pages 761-766, Monterey, November 1998; A. Toet, “New false color mapping for image fusion.” Optical Engineering, 35(3): 650-658, 1996; U.S. Pat. No. 5,325,449, P. Burt et al.; U.S. Pat. No. 5,555,324, A. Waxman et al.]. As the thermal infrared spectrum is beyond human visual perception, the merging of visible and thermal infrared modalities is a non-literal representation placing paramount importance on a visually intuitive fusion methodology. Furthermore, experience has shown that image fusion algorithms appearing intuitive and informative for single still frames are not necessarily appropriate under conditions of continuous operation, particularly if a human observer is immersed in such an environment for significant lengths of time. As with direct human vision, an image fusion scheme needs to have flexibility and adaptability to different scene conditions, where the visible and thermal infrared signatures can have a wide range of combined variability. The user should have the ability to change fusion parameters in order to optimize viewing conditions.